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ABSTRACT 


In recent years, a number of near midair collisions has shed light on the increased 
likelihood of mishaps that are partly attributable to traffic density. In air traffic control 
operations, situational awareness of a local controller at an airfield such as Marine Corps 
Air Station Camp Pendleton, California, is critical to prevention of catastrophic midair 
collisions. Spatialized audio technology has the potential to reduce or eliminate 
temporary losses of situational awareness. Spatialized audio technology allows auditory 
icons to be presented at perceptual locations external to the head at a complete range of 
elevation and azimuth locations relative to the listener. 

This thesis investigates the use of these spatialized auditory icons to determine if 
they could be effectively implemented in air traffic control type tasks to benefit local 
controllers. The research was conducted in the Advanced Auditory Displays Laboratory 
at NPS. A scenario to exemplify typical airfield operations at Camp Pendleton was 
written using java-based computer code. A virtual tower environment was created with a 
head mounted video display, inertial head tracker, and a spatialized audio server. Results 
indicated that subjects of the experiment responded more rapidly and accurately using 


spatialized audio. 
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EXECUTIVE SUMMARY 


In the 1990s, Congress’ Base Realignment and Closure Committees 
recommended the closure of numerous military installations, to include a number of 
military airfields. Airfields that remained operational experienced increases in traffic 
density due to the receipt of more tenant aircraft from the closed installations. Marine 
Corps Air Station (MCAS) Camp Pendleton, California, for example, received three H-46 
squadrons from the closures of MCAS Tustin and MCAS EI Toro. As a result, Camp 


Pendleton’s already “crowded” airspace became more congested. 


In recent years, a number of near midair collisions have shed light on the 
increased likelihood of mishaps that are partly attributable to traffic density. In air traffic 
control (ATC) operations, situational awareness of a local controller at an airfield such as 
Camp Pendleton is critical to prevention of catastrophic near midair collisions. 
Spatialized audio technology has the potential to reduce or eliminate temporary losses of 
situational awareness. Spatialized audio technology allows auditory icons to be presented 
at perceptual locations external to the head at a complete range of elevation and azimuth 
locations relative to the listener. This thesis implemented a prototype spatialized audio 
display and completed a performance evaluation to determine if this technology could be 
implemented in ATC settings. The prototype used a relatively inexpensive audio server, 
an inertial head tracking device, hi-fidelity headphones, and a head mounted display. 
Results from the evaluation show a significant increase in accuracy rates and response 
rates (degrees per sec of head turn), and a significant decrease in response times. The 
results also show that this system could be a cost-effective forerunner of a spatialized 
audio display system for military and civilian ATC. The prevention of one mid-air 
collision would more than pay for the cost of developing and implementing such a 
system. The cost-savings in human life is incalculable. However, the impact of this 
thesis reaches far beyond military and civilian ATC. It has applicability to unmanned 
aerial vehicle operations and manned aircraft operations as well. Additional research is 
planned to expand upon the research initiated in this project to further refine the process 


and deliver an actual product that can be implemented in the operational environment. 
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I. INTRODUCTION 


A. BACKGROUND 


High situational awareness levels are required of all air traffic controllers to 
maintain safe and efficient operating environments at the world’s airports and in the 
world’s airways. Information to sustain high situational awareness levels comes from 
audio communications, radar displays, and visual sighting of aircraft. In an effort to 
ensure minimum aircraft separation regulations are met, audio communications are most 
often used by air traffic controllers to pass directions or announce intentions of flight 
crews. Aircraft separation becomes an increasingly difficult task for an air traffic 


controller as the number of aircraft in the controller’s airspace grows. 


Losing situational awareness, even for a few seconds, can negatively impact the 
safety of the skies. Unfortunately there are many examples throughout aviation history 
where the absence of situational awareness has led to the loss of human life.! 
Undoubtedly there are countless other undocumented incidents where human life was 
luckily spared. One way of reducing or eliminating temporary losses in situational 
awareness is through the use of spatialized audio communications. In many instances, 
we believe that spatialized audio communications could have avoided misidentification 


of aircraft or eliminated the potential for a mid air collision. 


Advances in modern technology make it possible to synthesize spatial auditory 
cues over headphones. Prohibitively large and expensive speaker configurations are no 
longer necessary to produce a three-dimensional listener space. The headphones would 
allow air traffic controllers to perceive radio communications as if they emanated from 
the aircraft’s location in three-dimensional space. Controllers could take advantage of 
research that has been previously completed on radio speech intelligibility. From the 
1950s to the present day, researchers have demonstrated the beneficial effects of 
spatialized audio on radio speech intelligibility (e.g., see Thompson & Webster, 1953; 
Campbell, 2002). Improvements in radio speech response accuracy have also been 

1 As recently as July, a midair occurred between a Russian Tupolev 154 and a German Boeing 


757 cargo plane that resulted in the loss of 71 lives, including nine young children and 43 youngsters 
between the ages of 12 and 18 (CNN, 2002). 


1 


shown (Speith, Curtis, & Webster, 1954; Thompson & Webster, 1963; McKinley & 
Ericson, 1995). 


However, the impact of this thesis reaches far beyond military and civilian air 
traffic control (ATC). It has applicability to unmanned aerial vehicle operations and 
manned aircraft operations as well. Many of the same issues encountered in ATC 
operations will be encountered by UAV operators as the numbers of UAVs in the skies 
above battlefields are increasingly populated by combinations of manned and unmanned 
aircraft. This same technology can be applied to information management inside the 
cockpit as well. Other research shows that implementing spatialized audio displays in the 
cockpit can significantly reduce response times to missile threats (Shilling, Letowski & 
Storms, 2000). Despite the significance of findings from studies such as those mentioned 
previously, auditory displays are given minimal attention in many operational areas, 


including ATC. 


B. RESEARCH OBJECTIVE 


The purpose of this research is to examine the benefits of spatialized audio 
communications for air traffic controllers. In particular, it investigates the use of 
spatialized auditory icons to determine if they could be effectively implemented to 
improve situational awareness in air traffic control type tasks to benefit local controllers. 
An airfield local controller encounters an environment similar, yet distinctly different, 
from that of an en route air traffic controller. A local controller is required to acquire 
aircraft using visual means and audio communications, whereas an enroute air traffic 
controller (handling the enroute portion of a flight plan) uses a radarscope and audio 
communications. Research was conducted in the Advanced Auditory Displays 
Laboratory (AADL) at the Naval Postgraduate School (NPS). This research follows the 
groundwork laid by previous studies conducted in the AADL (e.g., see Campbell, 2002). 
A scenario to exemplify typical airfield operations at Camp Pendleton was written using 
extensive java-based computer code. A virtual tower environment was created with a 
head-mounted video display, inertial head tracker, and a spatialized audio server. An 


analysis of results from this experiment and previous studies could lead to new 
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hypotheses and testing for application of superior concepts in both civilian and military 
aviation and can be extended to situational awareness concerns for the operators of 
Remotely Piloted Vehicles (RPV) as well. The objective of nearly all research related to 
air traffic control is to improve the safety and efficiency of aviation operations. Starting 
in the 1980s, however, several events occurred that made the implementation of 
spatialized audio in a local airfield environment particularly more attractive to the U.S. 


military. 


C. IMPACT OF BASE REALIGNMENTS AND CLOSURES ON AIRFIELD 
OPERATIONS 


Between 1988 and 1995, the U.S. Department of Defense conducted four rounds 
of base realignments and closures (BRAC). The rounds were needed to reduce the 
nation’s domestic military basing infrastructure and bring it more in line with its force 
structure and funding levels. BRAC recommended the closure of a variety of 
installations to include military airfields. Airfields not selected by BRAC rounds were, 
nonetheless, affected by BRAC decisions. The airfields that survived experienced 


increases in traffic density due to the receipt of more tenant aircraft. 


One example of an airfield affected by BRAC is Marine Corps Air Station Camp 
Pendleton (MCAS Camp Pendleton). Prior to 1998, MCAS Camp Pendleton was home 
to six H-1 squadrons, possessing about 150 aircraft. By the end of 1999, Camp Pendleton 
inherited two H-46 tactical squadrons from the closures of MCAS EI Toro and Tustin. 
Additionally, in an effort to make room for its V-22 Fleet Replacement Squadron (FRS) 
at MCAS New River, North Carolina, the Marine Corps moved its H-46 FRS to MCAS 
Camp Pendleton. Once the helicopter transfers were complete, Camp Pendleton had 
acquired three H-46 squadrons and its total fleet rose to approximately 180 aircraft. It 
now owned nine squadrons, including the H-1 and H-46 FRS’s—squadrons that typically 
burden an air space more than “tactical” squadrons due to their inherent training 
missions. Camp Pendleton’s “crowded” airspace, which includes only one runway (6000 


feet in length), became more congested. 


D. IMPACT OF TRAFFIC DENSITY ON PROPENSITY FOR NEAR MID- 
AIR COLLISIONS 


In recent years, MCAS Camp Pendleton experienced a number of near mid-air 
collisions at Camp Pendleton. The near mid-air collision hazard reports that resulted 
from these events gave birth to improved flight routing and altitude restrictions (also 
known as course rules). Efforts to spread out flight operations more evenly throughout 
the day have also been implemented. Despite these steps, however, traffic density 


continues to be a problem at the airfield. 


A period of high traffic density poses a challenge to the local controller at an 
airfield such as Camp Pendleton. Local controllers are highly trained military and 
civilian air traffic controllers that are responsible for the safe separation of aircraft in the 
tower’s airspace. High traffic density correlates to more aircraft in the controller’s 
airspace. Thus the risk of a mid-air collision is increased. Although ultimate 
responsibility for aircraft avoidance lies with the aircraft commander (pilots), tower 
personnel can contribute greatly to safe operating conditions. Local controllers are 
human and subject to error just as aviators are. A common thread in recent near mid-air 
events at Camp Pendleton has been the temporary loss of the local controller’s situational 
awareness. The situational awareness of a local controller at an airfield remains critical 
to prevention of midair collisions, and even more so during inclement weather conditions. 
This is especially important at Camp Pendleton and other coastal airspaces that must deal 


with fog and marine layers on a consistent basis. 


E. CURRENT AUDIO TECHNOLOGY AND VISUAL ACQUISITION 
METHODS USED BY LOCAL CONTROLLERS 


Local controllers use a single earpiece/microphone device that provides a means 
of transmitting and receiving radio communications. The monaural audio heard in the 
blocked meatus earpiece and the controller’s eyesight are the primary means of acquiring 
aircraft in the tower’s airspace. Reducing the amount of time to accurately locate an 
aircraft once it transmits a radio call would have a favorable impact on a local 
controller’s situational awareness. As mentioned previously, one possible way to achieve 


this reduction is through the use of spatialized sound. To produce spatialized sound for 
4 


the local controller of an airfield, the azimuth and elevation of aircraft radio transmissions 
need to be determined. Currently, hardware does not exist to produce spatialized sound 
in a tower environment. Nonetheless, tools to enhance and maintain the controller’s 


situational awareness are worthy of attention. 


F. ORGANIZATION OF THIS STUDY 


This thesis is organized into six chapters. Chapter | offers initial motivation for 
pursuing the study of spatialized audio in an air traffic control environment. Chapter 2 
provides a basic background on spatialized audio, reviews literature that has applicability 
to air traffic control, and discusses literature pertinent to the design of the experiment 
used in the thesis. Chapter 3 describes the methodology behind the airfield tower 
simulation designed for this thesis. Chapter 4 analyzes the results of the tower 
simulation. Chapter 5 concludes the thesis and provides recommendations for future 


study. 
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Il. HUMAN LOCALIZATION, SPATIALIZED AUDIO, AND 
PREVIOUS STUDIES 


A. LOCALIZATION 


Localization refers to a human’s ability to make judgments as to the direction and 
distance of a sound source in the real world environment. Sound waves travel in all 
directions from a sound source. Thus, the listener must cope with direct and indirect 
sounds. Direct sound is the most “direct” path that sound travels to reach the listener 
(straight path). Indirect sound incorporates all of the reflections of the sound that a 
perceiver hears at a delayed time interval from the direct sound. Humans localize sound 
through the use of ears and the brain once the direct and indirect sound waves reach the 
listener. It is possible for humans to localize sound with one ear (for example, on a 
telephone), but this discussion will focus on binaural hearing. The two localization cues 
used by humans are interaural intensity difference (IID) and interaural time difference 
(ITD). IID refers to the fact that sound is louder at the ear to which it is closest. ITD 
refers to the fact that sound will arrive at one ear earlier than another. Before a sound 
wave enters the eardrum, it encounters the outer ear structure, or the pinnae. Pinnae act 
as a “filter” affecting every sound that passes through it. Following the pinnae are the 
effects of the ear canal (meatus), the middle ear, and finally the inner ear. Many 
literature sources provide a more detailed explanation of this process (e.g., see Begault, 


2000). 


Spatialized sound attempts to replicate the “filtering” process of the outer ear 
through the use of Head-Related Transfer Functions (HRTFs). Sound waves that reach 
the ears are affected by interaction with the listener’s torso, head, pinnae, and ear canals. 
Head-Related Transfer Functions (HRTFs) account for these acoustical effects by 
analyzing what happens to a single sound as it arrives to a listener from different angles. 
HRTFs can be thought of as two audio filters, one for each ear. Every person has a 
unique set of “individualized” HRTFs, but “generic” (or “non-individualized”) HRTFs do 
exist. HRTFs capture the information necessary to simulate a realistic sound space and 
produce spatialized sound. However, the measurement of individualized HRTFs can be a 


difficult, time-consuming process. Most audio systems use a standard set of HRTFs that 
‘| 


are not matched to an individual. Such was the case for the audio used in the tower 
simulation for this thesis. Participants did not have the opportunity for extensive spatial 
audio training. Even in the worst case, using non-individualized HRTFs does not 
degrade localization accuracy much more than the listener’s inherent ability (Wenzel, 


1992). 


B. SPATIALIZED AUDIO 


Spatialized audio is the product of modifying an audio signal for the purpose of 
making it appear to a human ear to originate from a source outside of a listener’s head. It 
essentially achieves a three-dimensional sound space outside a listener. 3-D sound can be 
delivered through headphones or loudspeakers that are placed around the listener. 
Literature indicates headphones are the medium of choice when control over the location 
of a spatial source is needed (Begault, 1999; Shilling & Shinn-Cunningham, 2002). 
Spatialized audio is not a diotic display (same signal to both ears) or merely a dichotic 
display (similar to diotic, but the source signal is delayed or scaled at one ear). Both 
diotic and dichotic displays are normally referred to as “lateralized” rather than 
“localized’ (Shilling & Shinn-Cunningham, 2002). The result of spatialized sound is 
audio that appears to emanate from 3-D locations, giving an additional cue for the 


location of particular objects. 


Research suggests that localization performance is optimized when the cues used 
in everyday hearing are reproduced as faithfully as possible (Begault, Wenzel, and 
Anderson, 2001). Available technology allows this reproduction to be achieved through 
the use of head trackers, HRTFs, and reverberation. Head trackers allow a sound source 
to remain in the same relative position to the listener despite his/her head movement. The 
purpose of HRTFs has been discussed. Reverberation replicates sound propagation and 
reflection in a particular environment. There have been several studies to identify the 


effect of any one of these techniques on sound localization performance. 


C; COMMON LOCALIZATION ERRORS 


Localization performance is measured by the occurrence of three common sound 
localization errors. These are localization error, externalization error, and reversal error. 


The following diagram depicts common errors. 


Front 


1: localization error— 
perceived source location deviates 
from exact position (azimuth or 
elevation) 

2: reversal error—source 
perceived on wrong side of 
interaural axis 

3: externalization error— 
source perceived as coming from 
inside head 


interaural axis 











Back 


Figure 1. Common audio localization errors 


D. IMPACTS OF HEAD TRACKING, INDIVIDUALIZED HRTF’S AND 
REVERBERATION 


Literature reveals many suggestions about the impact of head tracking, HRTFs, 
and reverberation. Not all of these suggestions concur with each other. Analysis of 
Variance (ANOVA) is used in nearly all experiments to see if any method (head tracking, 
individualized HRTFs, or reverberation) has a main effect on sound localization error 
rates. Individualized HRTFs are unique HRTFs collected on an individual. Generic 
HRTFs are used to suit a wider population that exhibits similar physical characteristics, 
such as head and pinnae size. A number of research studies in the 1980s and 1990s 
concluded that the use of individualized HRTFs reduced localization and reversal errors 


(Wenzel, Arruda, Kistler, and Wightman, 1993; Wightman, Kistler, and Perkins, 1987; 
9 


Moller, Sorensen, Jensen, and Hammershoi, 1995; Loomis, Hebert, and Cincinelli, 1990; 
and Mackensen, Fruhmann, Thanner, Theile, Horbach, and Karamustafaoglu, 2000). 
These same studies showed head tracking enhanced externalization while reducing 
localization errors. All research attempts strived to see if one of the methods reduced 


sound localization errors. 


More recently published research countered many previously held beliefs 
(Begault, Wenzel, and Anderson, 2001). Perhaps the most striking result was that head 
tracking and HRTF use had no significant effect on localization error rates. An 
interaction between the two did exist, but only for azimuth errors. Head tracking was the 
sole method found to have an impact on reducing reversal rates. Only reverberation was 
found to have an affect on externalization. This research was the first to compare the 
relative advantage of each method in a single experiment. The study used a virtual 
speech source—the source used in many applications of spatialized sound. Use of 


different sound sources may produce different results. 


E. IMPACTS OF SPATIALIZED SOUND ON VISUAL ACQUISITION 


Perrott, Cisneros, McKinley, and D’Angelo (1996) evaluated visual search 
performance in a free-field (anechoic) listening situation using both aurally aided and 
unaided cues. Not surprisingly, they found that search times were reduced with aurally 
aided cues. However, their overall approach to the problem of information overload was 
counterintuitive. Along with other researchers in the 1980s and 1990s, they recognized 
that “system operators” (pilots) suffered not from too much information but, rather, from 
too little information regarding which information source was most relevant at any one 
time. The researchers sought to show how spatial cueing could provide missing 
information to improve search performance. This type of problem is very analogous to 
the problems facing tower controllers at military airfields. The surprising part of the 
research was how much the search times were reduced with aural cues. Participants were 
seated in the center of a large sphere and given sound/visual cues from all azimuths and 
elevations. Search times of 1500 milliseconds (ms) were common for visual cues near 


the participant’s line of gaze. For those outside the line of gaze, times of 2000 to 3000 
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ms were common. With aural cueing, these times were reduced to 750 to 1000 ms in the 
line of gaze, 1250 ms outside the line of gaze. In other words, participants found their 
targets faster behind them with aural cueing than they found targets in front of them 
without cueing. Another interesting conclusion found no significant difference in 
response time using two (azimuth only) or three-dimensional sound. The study used 
“pink” noise (equal energy at each octave) of broad bandwidth as aural sources. This is 
not very applicable to radio communications, but the results were still impressive. Few 
distracters were used to force participants to decide between sources—participants had to 
either choose which light source was emitting sound, or count the total number of sources 
(two or three). This type of choice is only partially applicable to airfield tower operations 
that may have eight to ten detractors. In most cases, local controllers at an airfield know 
the vicinity of where an aircraft’s radio transmission will occur since approach patterns 
are highly structured. The Federal Aviation Administration deals with deviations from 


these established and published approach patterns harshly. 


F. SPATIALIZED SOUND STUDIES INVOLVING AVIATION AND AIR 
TRAFFIC CONTROL 


Since the 1990s, research has been ongoing at the Spatial Auditory Display 
Laboratory at NASA Ames Research Center in the area of "head-up auditory displays" 
for enhancing commercial aviation safety. Similar to a head-up visual display, the use of 
spatialized audio cues allows pilots to keep their visual gaze out-the-window "head-up" 
while simultaneously receiving situational awareness information via spatial audio. 
Crews of commercial airline crews were evaluated in a flight simulator to determine 
whether the existing Traffic alert and Collision Avoidance System (TCAS) could be 
improved by including spatial audio cues with the current head-down map display 
system. The experiment had verbal traffic alerts for TCAS advisories processed so their 
perceived spatial location corresponded to the location of aircraft traffic seen out of the 
windshield. Results indicated a significant difference for target acquisition time favoring 


the spatial audio TCAS condition by 307 ms on average (Wenzel & Begault, 1998). 


Additional research has been completed at NASA Ames on the Terminal Area 


Productivity (TAP) Program. A goal of the TAP program was to show how the 
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implementation of a 3-D Audio Ground Collision Avoidance System (GCAS) could 
improve the efficiency of airport surface operations for commercial aircraft operating in 
deteriorated weather conditions while maintaining a high degree of safety. The 3-D 
audio provided immediate awareness about crossing runways, potential conflicts with 
other aircraft, and when the aircraft deviated from centerline. Twelve crews were studied 
in a 747 simulator to see if a significant preference for an audio GCAS system existed on 
the part of the pilots. The main conclusions of the study were that an audio GCAS 
system would be useful for avoiding potential conflicts in both low-visibility and normal 
weather conditions (Wenzel & Begault, 1995). Another conclusion was that taxi times 


were not reduced significantly with 3-D audio cueing. 


New technologies for ATC systems have been under development for 
implementation in the Federal Aviation Administration’s Free Flight program. In the 
future, the aim of Free Flight is to allow pilots and airlines to set their own courses and 
resolve conflicts autonomously when possible. Even though pilots would be responsible 
for maintaining separation and awareness of immediate traffic, ATC would still be 
required to oversee separation assurance, intervene under emergency situations (e.g., 
failure of equipment aboard aircraft), and monitor the transition of flights to managed 
airspace (Metzger, 2001). To assist ATC, researchers have applied 3-D visualization and 
virtual reality technology to depict the zones around Free Flight aircraft. Spatial audio 
was then added to this technology. No user studies have been published to show the 
effect of adding spatial audio but user feedback was mostly positive. Controllers 
recommended using the spatialized audio features in training new controllers, to help 
them build a “mental model” of the 3-D nature of the airspace (Azuma, Daily, & Krozel, 
1996). A few suggested it might speed up transition time when changing controllers at a 
station, where the incoming controller must spend time monitoring displays to gain a 
better understanding of the 3-D situation before he can relieve the controller on duty 


(Azuma, Daily, & Krozel, 1996). 


A review of pertinent literature has shown that spatialized audio is beneficial in 
many environments, to include those that involve ATC. Upon the arrival of affordable 
technology that made it possible to synthesize spatial auditory cues over headphones, 


researchers recognized the potential of spatial auditory displays to enhance both 
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occupational and operational safety and efficiency of aeronautical operations (Begault, 
1993). The literature has also shown that localization performance is highly dependent 
upon a number of factors that include head tracking, HRTFs, reverberation, source and 


type of aural cues, type of task(s) required of the listener, and listening environment. 
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Hl. METHODOLOGY AND DATA COLLECTION FOR A 
SIMULATION INVOLVING A LOCAL AIRFIELD CONTROLLER 


A. PERFORMANCE MEASURES OF INTEREST FOR A SIMULATION 
INVOLVING A LOCAL AIRFIELD CONTROLLER 


Prior to discussing research methodology for this study, we first must identify our 
measures of interest. The measures of interest include the following: controller’s time to 
acquire an aircraft from initial aircraft transmission, controller’s average angular turning 
rate (degrees per second) from initial aircraft transmission, and accuracy of controller’s 
responses. In addition to spatialized audio factors mentioned in Chapter II, several 
additional factors need to be considered when analyzing the performance measures of 


airfield local controllers. 


B. ADDITIONAL FACTORS INFLUENCING PERFORMANCE MEASURES 
OF AIRFIELD LOCAL CONTROLLERS 


Factors affecting controller responses are not limited to whether or not localized 
sound is used. One factor is the number of aircraft in the pattern. If only one aircraft is 
in the pattern (performing multiple take offs and landings), it does not take a controller 
very long to recognize the direction of the radio call. However, if there are six aircraft 
flying within the controller’s airspace, identifying the source of a radio transmission 


becomes more challenging. 


Another factor that impacts the performance of a local controller is the location of 
the transmitting aircraft. If an aircraft is located directly in front of the controller, the 
response time to acquire the aircraft will be less than that required to locate an aircraft 
directly behind the controller because the controller must physically turn around to find 


the aircraft. 


Clarity and quality of a radio call play a factor. A poor quality radio call requires 
extra thought on behalf of the controller to interpret the radio call. Clear radio calls do 


not demand such extra effort and are responded to quicker. 
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Physical attributes and history of the controller play a part in aircraft acquisition. 
Quality of eyesight, experience level, training, and previous night’s sleep are examples of 


these factors. 


At first glance, the next item seems counterintuitive--it takes a controller longer to 
acquire an aircraft in the daytime than at night. Nighttime aircraft are typically easier to 
locate because of navigation lights installed on the aircraft. Aircraft manufacturers, for 
obvious reasons, intentionally design aircraft to be difficult to see during the day against 
certain types of backgrounds (sky, wooded, or desert camouflage, for example). Other 
meteorological conditions such as rain, fog, cloud cover, sun position, moon position, and 
moon phase all impact controller responses. In a real airfield tower environment, the 
aforementioned factors come into play. In our experiment, all aircraft were high-contrast 


objects against a blue sky, making them highly detectable with visual cues. 


In a simulated tower environment such as the one designed for this thesis, the 
artificiality of the scenario unavoidably comes into play. Learning effects occur in an 
experiment. Susceptibility to simulator sickness may affect results. If a particular 
airfield is modeled after an existing airfield, such as Camp Pendleton, participants may 
have experience with the airfield and will be familiar with flight operations. Most of the 
factors listed in this paragraph are considered uncontrollable “noise factors” for purposes 


of this research. 


C. PARTICIPANT COMPOSITION AND QUALIFICATIONS 


Thirty male graduate students and instructors from NPS were used in the virtual 
tower scenario. The participants were comprised of the following active duty U.S. 
military officers: 16 Marine Corps, 13 Navy, and 1 Army. Average age of participants 
was 35.3 years (SD 4.1 years). Very few students or instructors at NPS have air traffic 
control experience. Therefore, to understand the content of radio transmission 
throughout the simulation, participants were required to have either civilian or military 
flight experience (pilot or naval flight officer). Besides the limited availability of air 
traffic controllers, the use of aviators allowed them to subjectively evaluate the general 


utility of spatialized audio in an aviation environment via questionnaire. The average 
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flight experience of the participants was 1597 hours (SD 705 hours). The average time 
since last flight was 21.9 months (SD 16.7 months). No participant reported any 
significant hearing loss. Five of the participants had flown at Camp Pendleton. Sixteen 
of the participants received spatialized sound for the first half of the experiment and non- 
spatialized sound the second half. The remaining fourteen participants received the 
opposite treatment, non-spatialized sound followed by spatialized sound. The entire 
experiment, including time required for paperwork, lasted approximately one hour per 


participant. 


D. EXPERIMENT SETUP AND PROCESS 


Upon arrival at the AADL, participants were required to complete required NPS 
Institutional Review Board forms and a background questionnaire (Appendix A and 
Appendix B, respectively). Participants were shown all equipment that would be used in 
the experiment, which included the following hardware: Dell Dimension 8100 client 
computer with a 1.7 GHz Pentium IV and GeForce4 video card, AuSIM Gold Series 
spatialized audio server (connected to client via RS-232 port), Sennheiser HD 570 open 
headphones, InterSense InterTrax” inertial head tracker (connected to client via USB 
port), Olympus FMD-700 Eye-Trek head mounted display (connected to client via S- 


video cable), and a Logitech cordless optical mouse. 


Headphones were used to produce spatialized sound for better control over sound 
source location. Loudspeakers would be inappropriate in a tower environment because 
they would interfere with the normal staffing of personnel in the tower. Normal staffing 
consists of a ground controller, data controller, local controller, and a supervisor. All 
positions have a requirement to freely communicate with each other. Therefore, 
Sennheiser HD 570 open headphones were a viable choice for best spatial audio and task 
performance. As mentioned in Chapter II, it is important to note that generic HRTFs 
were used since collection of individualized HRTFs was precluded by time constraints on 


participants. 


Head tracking was employed through the use of an InterSense2 inertial head 


tracker mounted on the top of the Sennheiser headphones. The head tracker fed the 
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participant’s head position data to the AuSIM server for spatialized sound. Head position 
data was also used to properly update video imaging via the Olympus head mounted 


display (HMD). 


Sound sources for the audio were obtained from actual tower tapes recorded at 
Camp Pendleton in January 2002. Approximately 45 minutes in length, the cassette was 
used to produce over 50 individual radio transmissions from AH-1W, UH-1N, and CH- 
46E aircraft. The individual transmissions were transferred to digital format (44.1 KHz, 
16-bit, mono “wav” files) using Sonic Foundry Sound Forge 5.0 software and installed 


on the AuSIM server. 


Considerable effort went into scripting the aircraft events that would simulate a 
typical day at MCAS Camp Pendleton. An agent-based simulation model called the 
Multi-Agent Robot Swarm Simulation (MARSS) was used as the main computing 
software to script twenty-eight minutes of aircraft maneuvers (Dickie, 2002). The 
original purpose of MARSS was completely unrelated to this research, but the model 
served as a highly capable platform to produce video imagery of a virtual airfield 
environment and run a tower simulation. The java-based code possessed a “3D viewer” 
that could transform a 2-dimensional image into a virtual 3-dimensional environment. 
Figure 2 shows a screenshot of MARSS with a 1.11-megabyte (2018 x 2016) jpeg image 
of MCAS Camp Pendleton imported into the program. The 3-dimensional view from the 
tower is on the left, while the 2-dimensional view is on the right. The direction depicted 


in the 3-dimensional view is approximated by the dotted line in the 2-dimensional view. 


MARSS used the AuSIM Application Programming Interface Java Native 
Interface (API JNI) to implement spatialized sound by means of the AuSIM server. The 
head tracker used the API JNI, too, but required the use of AUAST_LT software available 
from AuSIM. AuAST LT is a C++ program that ran from the command line, external to 
MARSS. 


The scenario incorporated multiple aircraft takeoffs, approaches, and landings to 


Runway 21 and the right grass (an area adjacent to the runway). As is the case in real 
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Figure 2. MARSS screenshot of MCAS Camp Pendleton 


life, both left and right-hand traffic patterns were used. Ground controlled approaches 
(GCAs) to the airfield, low work (hovering) aircraft, and numerous entries and departures 
to the airfield were scripted. The following visual flight rules reporting points were used: 
North Initial, LCAC, VORTAC, TALA, delta taxiway, north grass, and south grass. To 
reduce the number of reporting points that participants were required to learn, Point 
Canyon, intersection, all taxiways except delta, the fuel pits, and the combined arms 
loading area (CALA) were not used. The minimum number of aircraft airborne at any 
one time was one helicopter. The maximum number of aircraft was six. All aircraft 
abided by local course rules and transmitted required radio calls at appropriate locations, 
such as “abeam for the grass” or “on the go.” Appendix C contains the individual radio 
calls heard by participants during the simulation. Before beginning the experiment, 
participants were exposed to sample radio calls to familiarize themselves with the 
differences between spatialized and non-spatialized audio, the quality of the audio 
transmissions, and “refresh” their memories of typical aircraft radio calls. The sequence 
of seven radio calls and one “inbound GCA” signal was played using the Sennheiser 
headphones with head tracking enabled. The participants did not wear the HMD for this 
introduction. Participants were also shown how to adjust volume levels (if needed) on 
the AuSIM server during this time. The “inbound GCA” signal replicated the sound 
produced by a console located in the tower of Camp Pendleton. The console allows the 
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local controller to electronically communicate with radar control to sequence instrument 
flight rules (IFR) aircraft into the tower’s visual flight rules (VFR) pattern. Radar 
controllers are not physically located in the same building as the tower controllers. For 
example, an aircraft on an instrument approach six miles from the airfield must have 
tower’s permission to enter its airspace. The radar controllers handling the aircraft 
transmit a signal to the tower that lights up a button on the console located in the tower. 
The local controller in the tower responds by pressing a button to notify the radar 
controllers that the aircraft is cleared through tower’s airspace (or other buttons that 
mandate a missed approach, wave off, full stop approach, etc.). Use of this console does 
not require either controller to talk on the telephone or communicate via radio, but these 


measures can be taken if necessary. 


Next, the participants were given a brief on the general layout of MCAS Camp 
Pendleton using the 2-dimensional view (see right hand side of Figure 2) of MARSS on 
the computer monitor. All reporting points mentioned previously were covered in this 
brief. They were also told of the aircraft that were presently in their airspace. 
Immediately following this brief the participants donned the HMD and headphones for 
another brief on the general layout of the airfield, this time in MARSS head tracked 3- 
dimensional viewer. Once the participants were comfortable and aware of the aircraft in 
their airspace, the cordless mouse was handed to the participant and the experiment 
began. To avoid visual distractions underneath the HMD’s field of view, the computer 
monitor was turned away from the participant’s line of sight. This also allowed the 
researcher to see exactly where the participant was looking since the view in the HMD 


and the client computer’s monitor were identical. 


When an aircraft transmitted, the participant of the experiment was asked to move 
their head in order to slew a circular transparent overlay to the immediate vicinity of the 
aircraft thought to be the source of the radio transmission. Immediate vicinity was 
defined as approximately twice the width of the overlay. The overlay was always located 
in the center of the participant’s field of view. The participant pressed the left mouse 
button once the overlay was on the aircraft. The cordless mouse was used to allow the 
participant to easily rotate in a swivel chair to acquire a “target.” At least twelve seconds 


elapsed between radio calls to allow the participant ample time to respond. Figure 3 
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shows a video image seen by participants wearing the HMD (and on the client computer 
monitor). Note the bull’s eye, or circular overlay, in the center of the image, and two 


aircraft, one each to the left and right of the image. 


alalx! 





Figure 3. HMD image of MCAS Camp Pendleton runway 


During the experiment, the researcher acted as the ground controller and gave 
appropriate warnings to the participant (the local controller) identical to those used at 
Camp Pendleton. For example, if an aircraft was taxiing to the hold short line on the 
delta taxiway for a VORTAC departure, the researcher called out, “Aircraft on Delta, 
VORTAC departure.” 


Each participant responded to sixty individual radio transmissions (approximately 
half were spatialized) during the 28-minute session. At the end of the simulation, 


participants completed the exit questionnaire located in Appendix D. 


E. DATA COLLECTION DURING THE SIMULATION 


As noted in the previous section, the researcher was able to see exactly where the 
participant’s line of gaze was located at any time during the experiment. Furthermore, 
the researcher could hear the click of the mouse button when the participant depressed it. 


Therefore, the researcher was able to determine if a participant identified the correct 
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aircraft in every response the participant made. The results were recorded on the 


response accuracy matrix located in Appendix E. 
Additionally, MARSS was programmed to record data throughout the experiment. 


For each participant, the data was written to a unique file upon completion of the 
experiment. Table 1 shows the data generated by MARSS. 








Simulation Participant ae eens oF ID & location 
Time Head Position : g of all aircraft 
aircraft 
Audio transmission by 
aircraft or GCA signal v v v 


Event 





Participant response my 
(mouse click) 














v v 


Data recorded by MARSS for specified events 








Table 1. 








From the recorded data, participant response time could be calculated as well as the total 


angular head displacement from aircraft transmission to participant response. 
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IV. DATA PREPARATION AND ANALYSIS OF RESULTS FROM 
THE LOCAL CONTROLLER SIMULATION 


A. DATA PREPARATION 


A significant amount of time was required to collate and prepare the data that was 
written to text files by MARSS. Numerous Microsoft Excel macros were composed to 
speed up the process and produce a data set that could be analyzed using analysis 
techniques available with S-Plus, a commercially available data analysis package. 
Appendix F contains mathematical formulas applied to the data to obtain response times 


and angular turning rates. 


The final data set consisted of 1,771 user responses. A total of 1,800 radio 
transmissions were presented, but the computer did not detect some participant responses. 
Other times, users either failed or forgot to respond to a radio call. Each of the 1,771 user 
responses had the following attributes as shown by a sample provided in Appendix G: 
subject number, subject treatment, radio call status, radio call position, response time, 
response rate, response accuracy, and identification number. Subject (participant) 
number identified the owner of the response (1-30). Subject (participant) treatment 
categorized the treatment received by the owner of the response (0 — non-spatialized 
sound followed by spatialized sound, 1 — spatialized sound followed by non-spatialized 
sound). Radio call status identified the aircraft audio transmission the user was 
responding to as either non-spatialized (0) or spatialized (1). Radio call position assigned 
the location of the radio call into one of seven areas (R — runway or right grass, A — right 
abeam, N — north initial, GCA, or short final for the runway, L— LCAC or VORTAC, LA 
— left abeam or TALA, H — hold short line, P — parallel taxiway). Response time was the 
number of seconds from aircraft transmission to user response (to tenths of a second). 
Response rate was the number degrees of head turn from aircraft transmission to user 
response, divided by response time (degrees per second). Response accuracy showed if 
the user response was correct (1) or not (0). Lastly, the identification number was the 


time of aircraft transmission (seconds). 
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B. EFFECT OF SPATIALIZED AUDIO ON ACCURACY RATES 


One of the primary areas to be evaluated in the local controller experiment was 
the effect of spatialized sound on the accuracy of subject responses. Of the 1,771 
responses recorded, 884 were taken during non-spatialized segments and 887 were from 
spatialized segments. 42 out of the total 1,771 responses were incorrect (32 non- 
spatialized and 11 spatialized). The average accuracies of both non-spatialized and 
spatialized statuses were quite high, 96.49% and 98.76%, respectively. However, 
analysis of a one-sided t-test helped determine that a significant difference existed 


between the two averages (p-value = 0.0009 as shown in Figure 4). 


Standard Two-Sample t-Test 


t = 3.1414, df = 1769, p-value = 0.0009 
alternative hypothesis: true difference in means is greater than 0 





sample estimates: 
mean of x mean of y 
0.9875986 0.9649321 





Figure 4. T-test results for difference in average accuracy rates between localized (x) 
and non-localized (y) responses 


The results of this test are important because they show a significant beneficial 
effect of spatialized sound on a participant’s situational awareness. This translates to 
fewer cases where an air traffic controller might inadvertently direct an aircraft in an 
unsafe manner—possibly resulting in the increased potential for a catastrophic mid-air 


collision and loss of valuable life. 


C; EFFECT OF SPATIALIZED AUDIO ON RESPONSE TIMES 


The next major item to be analyzed from the simulation was participant response 
times. The general trend of the data showed most participants tended to have faster 
reaction times using spatialized sound (see Appendix H). A t-test of the data set showed 
a significant difference between average non-spatialized audio responses versus 
spatialized audio responses. Figure 5 shows the results of this one-sided t-test (p-value < 
0.00001). 
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tandard Two-Sample t-1 





t = -5.6382, df = 1727, p-value = 0 
alternative hypothesis: true difference in means is less than 0 





sample estimates: 
mean of x mean of y 
4.401142 5.112896 








Figure 5. T-test results for difference in average response times between localized (x) 
and non-localized (y) responses 


A slightly more sophisticated approach to analyzing the effect of spatialized audio 
on response times was possible through the use of analysis of variance (ANOVA) 
techniques. Before this approach was taken, it was necessary to eliminate incorrect 
responses that were discussed in the previous section. Inclusion of incorrect responses in 
response times did not make logical sense, since a participant could simply click the left 


mouse button haphazardly to achieve extraordinarily brief response times. 


Prior to the application of ANOVA techniques, heteroscedasticity of the data was 
checked with a plot of participant mean response times versus standard deviations (Figure 


6). 


Mean vs. SD of Average Subject Response Times 
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Figure 6. Heteroscedasticity check for response times 
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This plot shows a general increasing linear trend to the data. Consequently, the 
logarithm (log) of response time was chosen as an appropriate method to reduce the 
effect of heteroscedasticity. Figure 7 shows the effect of taking the log of response 


times. 


Mean (log) vs. SD (log) of Average Subject Response Times 
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Figure 7. Heteroscedasticity check for log(response times) 


The behavior shown in Figure 7 is much more acceptable than the behavior shown in 


Figure 6. 


The first ANOVA model attempted to determine the effect of a number of 
potential factors that might influence response times. For a particular response, these 
factors included differences between participants (simply put, some participants will react 
faster than others), audio status (non-spatialized or spatialized), and position of radio call 
(one of the seven positions listed in section A of this chapter). Figure 8 shows the results 


of the initial attempt. 
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Analysis of Variance Table 


Response: log(resp.time) 


Terms added sequentially (first to last) 


Df Sum of Sq Mean Sq F Value Pr(F) 
as.factor(subject) 29 59.4980 2.05166 15.0693 0.00000 
status 1 7.8880 7.88799 57.9370 0.00000 
pos 6 103.4576 17.24294 126.6487 0.00000 
Residuals 1692 230.3620 0.13615 





Figure 8. Log of response time ANOVA output (main factors only) 


Not surprisingly, this chart shows that participants and positions have a significant 
impact on response times. Most importantly to this study, however, is the p-value less 
than 0.00001 associated with status. This means that spatialized audio had an effect on 
response times. We know this effect was a beneficial one by looking at the coefficient of 
“status” (-0.13) in the underlying linear regression model (Appendix I). The next step 
was to analyze the effects of interactions. The ANOVA output in Figure 9 shows the 


inclusion of interactions. 


Analysis of Variance Table 


DE Sum of Sq Mean Sq F Value Pr (F) 
as.factor (subject) 29 59.4980 -05166 15.8296 -000000 
status 1 7.8880 -88799 60.8599 - 000000 


pos 6 103.4576 ~24294 0380 -000000 
as.factor (subject) :status 29 7.7001 -26552 -0486 -000873 
as.factor (subject) :pos 174 29.6740 -17054 .3158 -005541 
Residuals 1489 192.9879 .12961 





Figure 9. Log of response time ANOVA output including interactions 


In addition to the three factors shown to have an effect in the previous ANOVA output, 
subject/status and subject/position interactions were shown to be significant (all p-values 
< 0.006). This was not unexpected since some participants seemed to respond to 


spatialized sound better than others. The same was true for a participant’s particular 
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ability to respond quicker to certain positions. A normal probability plot of the residual 
errors obtained from the underlying response time model verified the assumption of 


normality (Figure 10). 


Normal Probability Plot of Residuals from Response Time Model 








residuals(responseTimeModel) 











T T i} 
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Quantiles of Standard Normal 


Figure 10. Normal probability plot for response time model 


A status/position interaction was not chosen for analysis in the preferred ANOVA due to 
the lack of sufficient degrees of freedom at some locations (such as left abeam or north 
initial). Regardless, other ANOVA models showed no interaction between 


status/position (see Appendix I). 


Taking the analysis one step further to account for individual differences, position 
of aircraft, and statistical interactions has shown that response times are reduced using 
spatialized sound. The ANOVA and t-tests combined to strongly confirm the beneficial 


effect of spatialized audio in the local controller simulation. 
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D. EFFECT OF SPATIALIZED AUDIO ON RESPONSE RATES 


An analysis of the effect of spatialized audio on participant response rates 
followed the same methodology as the previous section’s analysis of participant response 
times. The general trend of the data showed most participants had relatively higher 
response rates during the spatialized audio segment of the experiment (see Appendix H). 
A closer look at the data, though, revealed some peculiarities in the data. For instance, 
why would it take one of our participants a total of 8.7 seconds to move his head 0.89 
degrees? There are a number of reasons how this could have happened. After the radio 
call, the participant may have looked away from his line of gaze to confirm the presence 
of other aircraft in the airspace before reacquiring the target. Or the participant may have 
been “leading” the aircraft with the bull’s eye in the HMD (analogous to a quarterback 
“leading” a receiver during a pass play in football), waiting for the aircraft to enter the 
overlay. Simply stated, interpretation of response rates is not as straightforward as 
response times. Nonetheless, a t-test of the data set showed a significant difference 
between average non-spatialized audio response rates and spatialized audio response rates 


(Figure 11, p-value = 0.0001). 


Standard Two-Sample t-Test 


t = 3.7356, df = 1727, p-value = 0.0001 
alternative hypothesis: true difference in means is greater than 0 





sample estimates: 
mean of x mean of y 
11.41584 9.765179 





Figure 11. T-test results for difference in average response rates between localized (x) 
and non-localized (y) responses 


These results lead one to believe that, on the average, participants move their heads faster 


toward transmitting aircraft when using spatialized cues. 


As was the case with response times, heteroscedasticity of the response rate data 
was checked with a plot of participant mean response rates versus standard deviations 


(Figure 12). 


Since heteroscedasticity was apparent, the log of the response rates was chosen as 


the appropriate method for countering heteroscedasticity. The effectiveness of a 
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logarithmic plot for response rates was not as effective as that for response times, but was 


still chosen (Figure 13). 


Mean vs. SD of Average Subject Response Rates 
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Figure 12. Heteroscedasticity check for response rates 


Mean (log) vs. SD (log) of Average Subject Response Rates 
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Figure 13. Heteroscedasticity check for log(response rates) 


30 


The need to eliminate one more data point became obvious when ANOVA 
models were attempted. One of the data points had a response rate of 0.000 deg/sec, so 
applying the log was not an option until that point was removed. Once this was done, the 


main effects were tested via ANOVA. The S-Plus output is listed in Figure 14. 


Analysis of Variance Table 


Response: log(resp.rate) 


Terms added sequentially (first to last) 
Df Sum of Sq Mean Sq F Value Pr (F) 
as.factor (subject) 29 93.114 3.21084 2.45490 0.0000276876 
status 1 14.333 14.33284 10.95838 0.0009514179 
pos 6 292.115 48.68575 37.22340 0.0000000000 
Residuals 1691 2211.716 1.30793 





Figure 14. Log of response rate ANOVA (main factors only) 


These results point out that participant, status, and position had an effect on response 
rates during the tower simulation (p-value = 0.00095). Next, an ANOVA with 
interactions was completed. The results of this ANOVA showed a significant interaction 


between participant and position (Figure 15). 


Analysis of Variance Table 
Response: log(resp.rate) 
Terms added sequentially (first to last) 


Df Sum of Sq Mean Sq Pr (F) 
as.factor (subject) 29 93.114 3.21084 ‘ -0000128 


status 1 14.333 14.33284 F .0007669 


pos 6 292.115 48.68575 F .0000000 

as.factor (subject) :status 29 41.850 1.44309 : -2728819 

as.factor(subject):pos 174 293.576 1.68722 : .0034754 
Residuals 1488 1876.291 1.26095 





Figure 15. Log of response rate ANOVA with interactions 


This interaction is not surprising for the same reason as noted in earlier sections—some 


participant’s possessed a particular ability to respond quicker to some positions. 
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Lastly, a normal probability plot of the residuals from the response rate model 


reveals a possible left tail (Figure 16). 


Normal Probability Plot of Residuals from Response Rate Model 





ie) 





residuals(responseRateModel) 











Ke} 4 ° 
o} 
T T T 
2 0 2 
Quantiles of Standard Normal 
Figure 16. Normal probability plot for response rate model 


The points making up the “curved” portion of the plot are relatively few when compared 
to the overall number. Arguments can be made from many directions about the 
plausibility of evaluating response rates in the manner that was done in this simulation. 
The fact of the matter is that response rates were highly dependent upon where a 
participant was looking at the time of a radio call. The location where a participant’s 
gaze was directed at the time of an aircraft’s transmission was dependent on factors 
beyond the scope of this thesis. Further analysis of response rates in this experiment was 
not warranted. Having said that, there still exists ample evidence to believe that 


spatialized sound had a positive impact on response rates during the experiment. 


E. EFFECT OF TREATMENT RECEIVED ON OVERALL RESULTS 


So far, the effect of treatment received by individual participants has not been 
discussed thoroughly. The inclusion of a treatment factor in ANOVA was infeasible 


since a participant only received one treatment. Were there differences in a participant’s 
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performance that depended upon treatment received? To find out, each participant’s 
average response time using spatialized sound was subtracted from the same participant’s 
average response time using non-spatialized sound. A t-test was then performed to see if 
the average difference for those that received non-spatialized audio followed by 
spatialized audio (treatment “‘x”) was different from those that received the spatialized 
audio first followed by non-spatialized audio (treatment “y”). The t-test results follow 
(Figure 17). 
Standard Two-Sample t-Test 


t = 2.3494, df = 28, p-value = 0.0261 
alternative hypothesis: true difference in means is not equal to 0 





sample estimates: 
mean of x mean of y 
0.3870498 1.011051 





Figure 17. T-test results for difference in average response rates between 
treatment x (non-3D > 3D audio) and treatment y (3D > non-3D audio) 


This t-test shows there is a difference (p-value = 0.0261). On the average, participants 
that received treatment “‘y” had a larger difference between average non-spatialized and 
spatialized response times than did those that received treatment “x.” In simple terms, 
one plausible reason for the difference was that spatialized audio made the aircraft 
acquisition process an inherently easier task to perform especially during the first half of 
the experiment. Those that received treatment “x” continued to exhibit the benefits of 
spatialized sound, only to a lesser extent because of familiarity gained from the first half 
of the experiment. Another plausible reason for the disparity between the treatments 
could be participant fatigue. During the second half of the experiment, a few participants 
reported mild eyestrain and/or the effects of a head tracked display on the vestibular 
system. There is no sure way to tell which case, if either, was the cause of the treatment 
effect. The only definitive conclusion that resulted from this portion of the analysis was 
that, on the average, spatialized sound helped participants acquire targets faster regardless 


of treatment received. 
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F. SUBJECTIVE FEEDBACK FROM PARTICIPANTS 


Each participant completed a questionnaire following the simulation that queried 
him about perceived levels of situational awareness throughout the experiment (Appendix 
D). A scale of 1 (worst) to 10 (best) was used. 26 of 30 participants rated their 
situational awareness during the spatialized portion of the experiment higher than that 
during the non-spatialized portion. Three participants rated them equal. One participant 
felt that his situational awareness was higher during the non-spatialized portion because 
he “had to actually listen to the content of the radio call.” The average rating of 
situational awareness during spatialized sound was 8.63. The average rating for non- 
spatialized sound was 6.17 (see Appendix J). In no case did a participant feel spatialized 
sound was a bad idea. Some participants came out of the experiment quite enthusiastic 
about the potential of spatialized sound in the tower, as well as in the potential 


applicability in the cockpit (see Appendix K). 
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Vv. CONCLUSIONS AND RECOMMENDATIONS 


A. ANALYTICAL CONCLUSIONS 


This thesis investigated the use of spatialized auditory icons and radio 
communications to determine if they could be effectively implemented in air traffic 
control tasks to benefit local controllers. The research provides quantitative evidence that 
spatialized audio helps to improve the accuracy of a local controller’s identification and 
acquisition of aircraft that are communicating with the tower. It also provides 
quantitative evidence that spatialized audio helps to reduce the amount of time required 
by the local controller to visually locate an aircraft emitting a radio transmission. Lastly, 


it shows that local controllers exhibit faster head turn rates using spatialized audio. 


B. IMPACT OF THIS STUDY 


The impact of this thesis reaches far beyond military and civilian ATC. It has 
applicability to remotely piloted vehicle (RPV) operations and manned aircraft operations 
as well. It is hypothesized that this technology can be used to increase the situational 
awareness of pilots for mental and physical perceptions of other aircraft in their airspace. 
Judging from the responses of participants that took part in the experiment, the question 
to be answered is not, “Should we have spatialized sound in the cockpit,” but rather, 


“Why don’t we have spatialized sound in the cockpit?” 


€; RECOMMENDATIONS FOR FUTURE STUDY 


The prototype used in this research used a relatively inexpensive audio server that 
can supply audio for four or more independent listeners. The study implemented an 
inexpensive inertial headtracking device and HMD. The analysis shows that this system 
could be a cost-effective forerunner of a spatialized audio display system for military and 
civilian air-traffic control. The prevention of one mid-air collision would more than pay 
for the cost of developing and implementing such a system. The cost-savings in human 


life is incalculable. 
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Finally, this thesis significantly impacts future research to be conducted at NPS 
and the Operations Research Department. Additional research is planned to expand upon 
the research initiated in this project to further refine the process and deliver an actual 
product that can be implemented in the operational environment. To avoid anomalies 
encountered during the analysis of head turn rates, it is recommended that future studies 
focus on the initiation of head turn rates (i.e., number of degrees of head turn in the first 
two seconds following a radio call). If possible, the HMD should be replaced with a full 
360-degree visual display. To this end, additional research funding is being sought to 
supplement the research initiated here and to develop specific requirements documents to 
drive the Small Business Innovation Research (SBIR) process for delivering a product to 


the operational world. 
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APPENDIX A. NPS INSTITUTIONAL REVIEW BOARD FORMS 


The following pages contain all forms that were completed by each participant 
prior to the commencement of the airfield local controller experiment. They include a 
Participant Consent Form, Minimal Risk Consent Statement, and a Privacy Act 


Statement. 


a4 


NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PARTICIPANT CONSENT FORM 


Introduction. You are invited to participate in a study of the situational awareness of 
air traffic controllers aboard a military airfield. With information gathered from you 
and other participants, we hope to evaluate the effectiveness of localized sound on 
situational awareness. We ask you to read and sign this form indicating that you agree 
to be in the study. Please ask any questions you may have before signing. 


Background Information. This study is part of ongoing effort on behalf of naval 
aviation, and aviation in general, to enhance safety and efficiency at airfields. The 
study is also a contributor to the Human Systems Integration Laboratory’s efforts to 
investigate virtual displays. 


Procedures. If you agree to participate in this study, the researcher will explain the 
tasks in detail. There will be two fifteen-minute sessions: 1) a simulation using 
localized sound and 2) a simulation using non-localized sound. The order of the 
simulations will vary from participant to participant. Overall time of the study will be 
approximately forty-five minutes to one hour. 


Risks and Benefits. This research involves no risks or discomforts greater than those 
encountered by playing an ordinary video game. The benefits to the participants are 
exposure to techniques for enhancing spatial knowledge in an aviation environment. 


Compensation. No tangible reward will be given. A copy of the results will be 
available to you at the conclusion of the experiment. 


Confidentiality. The records of this study will be kept confidential. No information 
will be publicly accessible which could identify you as a participant. 


. Voluntary Nature of the Study. If you agree to participate, you are free to withdraw 
from the study at any time without prejudice. You will be provided a copy of this form 


for your records. 


Points of Contact. If you have any further questions or comments after the 


completion of the study, you may contact the research supervisor, Dr. Russell Shilling, 
(831) 656-2543 rdshilli@nps.navy.mil. 





8. Statement of Consent. I have read the above information. I have asked all questions 


and have had my questions answered. I agree to participate in this study. 


Researcher’s Signature Date 
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NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
MINIMAL RISK CONSENT STATEMENT 


Participant. VOLUNTARY CONSENT TO BE A RESEARCH PARTICIPANT IN: 
Enhancing the Situational Awareness of Local Airfield Controllers 


1. 


I have read, understand and been provided "Information for Participants" that provides the 
details of the below acknowledgments. 


I understand that this project involves research. An explanation of the purposes of the 
research, a description of procedures to be used, identification of experimental procedures, 
and the extended duration of my participation have been provided to me. 


I understand that this project does not involve more than minimal risk. I have been informed 
of any reasonably foreseeable risks or discomforts to me. 


I have been informed of any benefits to me or to others that may reasonably be expected from 
the research. 


I have signed a statement describing the extent to which confidentiality of records identifying 
me will be maintained. 


I have been informed of any compensation and/or medical treatments available if injury 
occurs and is so, what they consist of, or where further information may be obtained. 


I understand that my participation in this project is voluntary; refusal to participate will 
involve no penalty or loss of benefits to which I am otherwise entitled. I also understand that 
I may discontinue participation at any time without penalty or loss of benefits to which I am 
otherwise entitled. 


I understand that the individual to contact should I need answers to pertinent questions about 
the research is Dr. Russell Shilling, Principal Investigator, and about my rights as a research 
participant or concerning a research related injury. A full and responsive discussion of the 
elements of this project and my consent has taken place. 


Medical Monitor: Flight Surgeon, Naval Postgraduate School 











Signature of Principal Investigator Date 
Signature of Volunteer Date 
Signature of Witness Date 
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NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PRIVACY ACT STATEMENT 


1. Authority: Naval Instruction 


2. Purpose: Situational awareness data will be collected to enhance knowledge, and to 
develop tests, procedures, and equipment to improve the development of Virtual 
Environments. 


3. Use: Situational awareness data will be used for statistical analysis by the 
Departments of the Navy and Defense, and other U.S. Government agencies, 
provided this use is compatible with the purpose for which the information was 
collected. The Naval Postgraduate School in accordance with the provisions of the 
Freedom of Information Act may grant use of the information to legitimate non- 
government agencies or individuals. 


4. Disclosure/Confidentiality: 


a. I have been assured that my privacy will be safeguarded. I will be assigned a 
control or code number, which thereafter will be the only identifying entry on any of 
the research records. The Principal Investigator will maintain the cross-reference 
between name and control number. It will be decoded only when beneficial to me or 
if some circumstances, which are not apparent at this time, would make it clear that 
decoding would enhance the value of the research data. In all cases, the provisions 
of the Privacy Act Statement will be honored. 


b. I understand that a record of the information contained in this Consent Statement 
or derived from the experiment described herein will be retained permanently at the 
Naval Postgraduate School or by higher authority. I voluntarily agree to its 
disclosure to agencies or individuals indicated in paragraph 3 and I have been 
informed that failure to agree to such disclosure may negate the purpose for which 
the experiment was conducted. 


c. I also understand that disclosure of the requested information, including my 
Social Security Number, is voluntary. 





Signature of Volunteer Name, Grade/Rank (if applicable) DOB SSN Date 





Signature of Witness Date 
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APPENDIX B. BACKGROUND QUESTIONNAIRE 


Subject Number: 


Age: Sex: Service: 
Years in service: Branch/MOS: 


Do you have any aviation or air traffic control related experience? 


If yes, please briefly explain here (military / civilian, type aircraft, number of 
hours, last flight, and air traffic control qualifications): 














Do you have any significant hearing loss? 


If yes, explain here: 














Circle the number that best estimates your video game ability: 


1 — Poor (What’s Doom, Asteroids, or Space Invaders?) 
2—Below Average (My kids have PlayStation 2, Game Cube, or X-box, but I rarely 
play [be honest]) 


3 — Average (I can hold my own) 
4— Above Average (I’ve read game manuals to learn special tricks) 
5 — Expert (Carpel Tunnel Syndrome is a certainty) 


Circle any of the following symptoms you are presently experiencing: 


Stomach awareness Yawning Burping 
Headache Eye strain Nausea 
Difficulty focusing General body fatigue Dizziness 
I feel like a champ 
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APPENDIX C. RADIO CALLS 


This appendix contains the time, file name and message content of all radio calls 


heard by participants during the airfield local controller experiment. 


p Time | Audio File Message 
55.0 
277.0 
321.9 
334.8 
"Campen Tower, Knightrider 00 is at the LCAC, inbound, landing full stop" 
398.2 
"Tower, Atlas 22 on the go for the right downwind" 
433.4 'Knightrider 00 approaching abeam, full stop" 
"Campen Tower, Atlas 49 number 1 hold short delta for 21, right downwind" 
"T las 32 is a single huey from the TALA for local bounce" 


las 35 is on the go, right grass" 

las 32 is abeam for duty" 

las 22 abeam for duty" 

las 49 is abeam, right grass" 

‘Tower, Atlas 32 on the go, duty" 

Tower, Atlas 35 request extended downwind for stuck pedals" 
jas49OnTheGoRtGrass.wav Ti las 49 is on the go, right grass" 
jas35TurningBase.wav Ti las 35 turning base" 

jas32OnTheGoRtGrass.wav T las 32 is on the go, right grass" 
jas350nTheGoDuty.wav T las 35 on the go from the duty" 

ift04DeltaVortac.wav ‘Tower, Swift 04 is number one hold short, delta, for a VORTAC departure" 


7 las 49 is abeam, right grass” 
on las 32 is abeam, right grass” 
892.1|Atlas35AbeamDuty.wav "Atlas 35 abeam, duty" 
"Campen Tower, Swift 04 is VORTAC for the switch" 
"Tower, Atlas 49 is on the go, right grass" 
"Campen Tower, Knightrider 03 takeoff, left downwind" 
975.6|Atlas32AbeamDuty.wav "Tower, Atlas 32 is abeam for duty" 
"Campen Tower, Swift 12 on the parallel, hover checks, north grass" 
1051.1 "Tower, Atlas 49 is abeam, right grass" 
1075.4 "none (sequence of five audio beeps)" 
1082.9 "Tower, Atlas 68 papa for the south grass, ten minutes" 
1108.3 
1120.3 "Tower, Viper 34 GCA handoff for local bounce" 
1138.1 "Tower, Atlas 49 is on the go, right grass" 
Viper 34 abeam, practice auto, grass” 
"Tower, Atlas 49 is abeam, right grass” 
"Tower, Viper 34 on the go, right grass" 
"Tower, Knightrider 03 is left abeam, practice auto" 
"Tower, Atlas 49 takeoff, right grass" 
Tower, Knightrider 03 is on the go" 
"Viper 34 abeam, practice auto, grass" 
"Tower, Atlas 49 is abeam for runway 21" 
"Campen Tower, Dragon 00 flight of four, LCAC inbound request extended downwind for the break" 
"Tower, Knightrider 03 is left abeam, practice auto" 
Knightrider03TakeoffVortac. wav 
"Campen Ground (Tower), Swift 12 is low work complete " 





























1667.7|Dragon0OLakeBreak.wav "Tower, Dragon 00 and flight is Lake O'Neill inbound, request mid field break" 
1705.5] Viper34Abeam21.wav "Viper 34 is abeam for 21" 








1715.5] KnightriderO3VortacClear.wav ower, Knightrider 03 is VORTAC clear" 
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APPENDIX D. EXIT QUESTIONNAIRE 


Subject Number: 


1. During the spatialized sound portion of the experiment, rate your overall situational 
awareness. 


Low Medium High 
0 1 2 3 4 5 6 7 8 9 10 


2. During the non-spatialized sound portion of the experiment, rate your overall 
situational awareness. 


Low Medium High 
0 1 2 3 4 5 6 7 8 9 10 


3. How well were you able to understand the audio transmissions of the aircraft? 


None Some Very Well 
0 I 2 3 4 5 6 7 8 9 10 


4. Circle any of the following symptoms you are presently experiencing: 
Stomach awareness Yawning Burping 
Headache Eye strain Nausea 
Difficulty focusing General body fatigue Dizziness 
I feel like a champ 

5. Any additional comments: 
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APPENDIX E. RESPONSE ACCURACY MATRIX 


This matrix shows the accuracy of each participant’s response during the airfield 


local controller experiment. Columns labeled 1-30 correspond to participant number. 


T= 3d --> non-3d F = non-3d --> 3d 

F T FI] TF 

Fea linen] efor] eneralinar | spel7/s 

HH 28.1 EA jas22 
2 = 27.7 Atlas35 

| 4 | 173.4] 25.7] A [Atlas35 | 


5 225.4| 24.8 Atlas22 

6 260.4| 24.3 Atlas35 

7 277| 24.0 Coyote32 
| 8 | 321.9] 23.2] N [Coyote32_ | 
| 9 | 334.8] 23.0] A [Atlas22 | 

10 377.9| 22.3] L_ |Knightrider0O 

11 398.2] 22.0] A |Atlas35 

12 | 421.8] 21.6] R_ |{Atlas22 


433.4| 21.4 A _|Knightrider0o 
447.3[ 21.1[ H [Atlas49 | 


15 | 480.3] 20.6] LA |Atlas32 
16 512.5] 20.0] R_|Atlas35 
17 532.2] 19.7] LA |Atlas32 


18 | 559.6] 19.3] A [Atas22 | 


19 [589.8] 18.8] A JAtlas49 | 
20 | 628.2| 18. 


21 648.4 
22 676.8 
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48 | 1390.2] 5.4] R [Atlas49 | 
49 


50 | 1483.1 3a] A pers 
51 | 1513.9] 3.4] A [Atlas49 
52 | 1537.9 3.0[ L [DragondoDashi{ [TT [ [ [| 
53 | 1562.6[2.5[ LA [KnightriderOs [ [TT [ [ J | 
54 [1597.41 2.0[ R_[Viper34_ 
55 | 1615.1 17|_A_[Pregontnpash 
56 | 1627.8] 1.5[ R_|[Knightrider03 

157 [1651.1] 1.1] P [Switt2 | 
| 58 | 1667.7] 0.8] N_ [Dragend0Dash+ | 


Psa[1705;5[0.2| A |Vipersa | 
bao tirisst oat feegneaans ft tf -+ tJ 














AIA 
Oy 





















































Positions 
R = runway or right grass_H = hold short at Delta bold = ground controller notification required for next aircraft call D = did not respond 
A= right abeam L =LCAC or VORTAC strikethrough = aircraft is full stop this approach X = mistake made 


LA = left abeam or TALA P = parallel taxiway 
N = north initial, short final, or GCA 
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APPENDIX F. APPLICABLE MATHEMATICAL FORMULAS 


The following equations were used to produce response times and response rates 


from the data collected during the experiment. 


RESPONSE TIME = time of user response — time of aircraft transmission 


For example, an aircraft begins its transmission at time t; = 55.0 seconds and the 


participant responded at time tz = 60.0 seconds, then 


RESPONSE TIME = t, — t; = 60.0 — 55.0 = 5.0 seconds. 





RESPONSE RATE = # of degrees of head turn / RESPONSE TIME 


The # of degrees of head turn (theta) was a little more complex to compute. 
Calculus was used to determine the angle, theta, between the head position at the time of 
aircraft transmission and the head position at the time of user response. 


theta = cos" [(A°B) / (|A| |B))] where “ *” represents a dot product, 
and |A| = magnitude of the vector A. 


The equation reduces to the following if the magnitudes of A and B are equal to 1: 
theta = cos (A ‘B) 


For example, let az, and elev, equal the azimuth and elevation, respectively, of 
the user’s head at the time of an aircraft transmission. Then vector A is determined by 
these formulas (in right-handed X, Y, Z coordinate system): 


X component = -cos(eleva,) * sin(aza), 
Y component = sin(eleva) 
Z component = -cos(eleva) * cos(aza) 


Similarly, these formulas apply to determine the vector that represents the head position 
at the time of user response as well. Finally, the trivial calculation cos ‘(A ° B) 


determines the angle theta. 
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APPENDIX G. SAMPLE DATA 


This appendix is an example of the data gathered from participant #1 during the 
airfield local tower scenario. Inclusion of all experimental data would be quite lengthy. 
For a complete data set, please contact Dr. Russell Shilling at the Naval Postgraduate 


School, (831) 656-2543. 


subject treatment status resp time pos resp angle resprate accuracy id 
1 1 1 1.9 R 37.9476  19.97242 1 55 
1 1 1 1.6 A 6.118595 3.824122 1 142.4 
1 1 1 0.6 A 2.482774 4.137957 1 173.4 
1 1 1 2.8 R 3.60466 1.287379 1 421.8 
1 1 1 3.1 A 12.55633 4.05043 1 433.4 
1 1 1 2.4 H 14.31875 5.966146 1 447.3 
1 1 1 3 T 74.43727 24.81242 0 480.3 
1 1 1 2.9 R 81.55037 28.12082 1 512.5 
1 1 1 6.5 LA 122.7071 18.87802 1 532.2 
1 1 1 1.1 A 1.784293 1.622085 1 559.6 
1 1 1 1.3 A 9.085597 6.988921 1 589.8 
1 1 1 2.9 R 51.31325 17.69422 1 628.2 
1 1 1 2.6 A 13.40196 5.154601 1 648.4 
1 1 1 1.9 R 31.12465 16.38139 1 676.8 
1 1 1 2.9 N 4.684776 1.61544 1 694.2 
1 1 1 0.7 A 3.279536 4.685051 1 737.6 
1 1 1 3 R 6.814918 2.271639 1 763.3 
1 1 1 1.1 H 1.704386 1.549442 1 779.2 
1 1 1 1.8 A 57.16414 31.75785 1 814.6 
1 1 1 2.1 R 80.6997  38.42843 1 851.9 
1 1 1 2.5 A 8.055538 3.222215 1 892.1 
1 1 1 5.1 L 62.36669 12.22876 1 909.6 
1 1 1 2.7 R 81.9266  30.34319 1 931.3 
1 1 0 3.3 H 4.478749 1.357197 1 952.2 
1 1 0 2.9 A 74.12908 25.56175 1 975.6 
1 1 0 4.8 P 80.52682 16.77642 1 1009.7 
1 1 0 0.8 A 0.612634 0.765793 1 1051.1 
1 1 0 5.9 N 153.951 26.09339 1 1075.4 
1 1 0 6.3 P 114.2211 18.13034 1 1082.9 
1 1 0 5.2 LA 132.6342 25.50657 1 1108.3 
1 1 0 4.9 S 128.9458 26.31548 1 1120.3 
1 1 0 1.2 A 5.804066 4.836722 1 1258.3 
1 1 0 2.2 A 12.49496 5.679529 1 1275.9 
1 1 0 2.6 R 19.94666 7.671794 1 1345.3 
1 1 0 5.9 A 108.501 18.38999 1 1483.1 
1 1 0 3.7 A 19.38602 5.239465 1 1513.9 
1 1 0 6.2 L 42.46573 6.849312 1 1537.9 
1 1 0 2.4 R 35.9508 14.9795 1 1597.4 
1 1 0 15.3 R 162.5618 10.62496 1 1600.9 
1 1 0 4.6 A 147.4996 32.06513 1 1615.1 
1 1 0 1.9 N 14.64398 7.707356 1 1667.7 
1 1 0 3.6 N 87.50718 24.30755 1 1689.6 
1 1 0 1.7 L 18.48977 10.87633 1 1721.1 
1 1 0 2.3 L 32.83853 14.27762 1 1777 
1 1 0 4.2 L 15.23521 3.627431 1 1786.9 


N 
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APPENDIX H. AVERAGE RESPONSE TIME AND RATES 


The table below lists average response times and rates for each participant. It is 
important to keep in mind that every participant did not have the same number of 
responses recorded during the experiment. For example, participant 1 only had 45 
responses, participant 2 had 59, and participant 3 had 58. These averages are only to 


show the reader general trends of the data. 


[Subject] non-3d [ 3d | nonad [ 3d | 
4.993 
6.186 
5.148 
4.697 
5.519 
3.783 
4.981 
5.348 
7.061 
6.189 
6.326 
4.983 
5.074 
5.448 
4.352 
6.221 
4.048 
7.504 
5.465 
8.179 
4.687 
4.700 
4.229 
4.769 
5.768 
3.341 
4.913 
5.204 





| 3 | 4.993 
| 4 | 6.186 
| 5 | 5.148 
| 6 4.697 
| 7 5.519 
| 8 3.783 
| 9 | 4.981 
| 10 | 5.348 
| 17.061 
| 12) | 6.189 
| 13 | 6.326 
| 14 | 4.983 
| 15 | 5.074 
| 16 | 5.448 
| ty | 4.352 
| 18 | 6.221 
| 19 | 4.048 
| 20, | 7.504 
| 21 | 5.465 
| 22 | 6.179 
| 23 | 4.687 
| 24 | 4.700 
| 25 | 4.229 
| 26 | 4.769 
| 27 | 5768 
| 28 | 3.344 
| 29) | 4.913 
| 30 5.204 
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APPENDIX I. S-PLUS DATA ANALYSIS OUTPUT 


This appendix contains S-Plus output for each t-test and AVOVA model. Linear 
regression models are also included to fully interpret the ANOVA models. For reference, 
status = 0 is non-spatialized sound, status = 1 is spatialized sound, 
treatment = 1 1s spatialized>non-spatialized, and treatment = 2 is non- 


spatialized> spatialized. 


One-sided t-test for difference in accuracy rates 


> t.test (sam2Saccuracy[sam2Sstatus == 1], 
sam2Saccuracy[sam2Sstatus == 0], alt = "greater") 


Standard Two-Sample t-Test 


data: sam2Saccuracy[sam2Sstatus == 1 
sam2Saccuracy[sam2Sstatus == 0 

t = 3.1414, dt = 1769, Seo 

alternative hypothesis: true difference in means is greater than 0 

sample estimates: 

mean of x mean of y 


i 





One-sided t-test for difference in response times 


> t.test (sam2Sresp.time[sam2Sstatus == 1], 
sam2Sresp.time[sam2Sstatus == 0], alt = "less") 


Standard Two-Sample t-Test 


data: sam2Sresp.time[sam2Sstatus == 1] and 
sam2Sresp.time[sam2Sstatus == 0] 
t = =5.6382, df = 1727, 


alternative hypothesis: true difference in means is less than 0 
sample estimates: 
mean of x mean of y 
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ANOVA model for main effects of log(response time) 














> anova (lm (log(resp.time) ~ as.factor(subject) + status + pos, 
data = sam2) ) 
Analysis of Variance Table 
Response: log(resp.time) 
Terms added sequentially (first to last) 
D£ Sum of Sq Mean Sq F Value Pr (F) 
as.factor (subject) 29 59.4980 2.05166 15.0693 0.000000e+000 
status 1 7.8880 7.88799 57.9370 
pos 6 103.4576 17.24294 126.6487 0.000000e+000 
Residuals 1692 230.3620 0.13615 
Regression model for main effects of log(response time) 
*** Linear Model *** 
Call: lm(formula = log(resp.time) ~ as.factor(subject) + status + pos, 
data = sam2) 
Residuals: 
Min 10 Median 30 Max 
-2.308 -0.2103 —-0.03184 0.201 1.736 
Coefficients: 
Value Std. Error t value Pr(>/tl) 
(Intercept) 0.8121 0.0577 14.0834 0.0000 
as.factor (subject) 2 0.3421 0.0735 4.6528 0.0000 
as.factor (subject) 3 0.3894 0.0750 5.1922 0.0000 
as.factor (subject) 4 0.6495 0.0744 8.7347 0.0000 
as.factor (subject) 5 0.5833 0.0735 7.9326 0.0000 
as.factor (subject) 6 0.2781 0.0741 3:2 FS25 0.0002 
as.factor (subject) 7 0.4183 0.0741 5.6462 0.0000 
as.factor (subject) 8 0.2175 0.0733 2.9682 0.0030 
as.factor (subject) 9 0.4961 0.0735 6.7464 0.0000 
as.factor (subject) 10 0.4706 0.0733 6.4219 0.0000 
as.factor (subject) 11 0.7968 0.0738 10.7957 0.0000 
as.factor (subject) 12 0.6267 0.0738 8.4898 0.0000 
(output edited for space) 
as.factor (subject) 29 0.5345 0.0736 7.2663 0.0000 
as.factor (subject) 30 0.5053 0.0741 6.8188 0.0000 
status 0.0178 -7.3269 
posH 0.2934 0.0425 6.8953 0.0000 
posL 0.6312 0.0374 16.8640 0.0000 
posLA 0.7246 0.0356 20.3279 0.0000 
posN 0.3585 0.0305 LL F379 0.0000 
posP 0.6924 0.0432 16.0233 0.0000 
posR 0.1794 0.0222 8.0932 0.0000 


Residual standard error: 0.369 on 1692 degrees of freedom 


F-statistic: 34.86 on 36 and 1692 degrees of freedom, the p-value is 0 
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ANOVA model showing no status:position interaction effect in log(response time) 


> anova (lm (log(resp.time) ~ as.factor(subject) * status + pos + 
status:pos, data = sam2) ) 
Analysis of Variance Table 


Response: log(resp.time) 


Terms added sequentially (first to last) 


Df Sum of Sq Mean Sq F Value Pr (F) 

as.factor (subject) 29 59.4980 2.05166 15.3396 0.0000000 

status 1 7.8880 7.88799 58.9761 0.0000000 

pos 6 103.4576 17.24294 128.9201 0.0000000 

as.factor (subject) :status 29 7.7001 -26552 1.9852 0.0014158 


0 
status:pos 6 1.0398 0.17329 1.2957 
Residuals 1657 221.6221 0.13375 


Final ANOVA model for main effects and interactions of log(response time) 


> anova (1m (log (resp.time) a as.factor (subject) - status + 
as.factor(subject)*pos, data = sam2) ) 
Analysis of Variance Table 


Response: log(resp.time) 


Terms added sequentially (first to last) 


Df Sum of Sq Mean Sq F Value Pr(F) 
as.factor (subject) 29 59.4980 2.05166 15.8296 0.000000000 
status 1 7.8880 7.88799 60.8599 
pos 6 103.4576 17.24294 133.0380 0.000000000 
as.factor (subject) :status 29 7.7001 -26552 2.0486 0.000873557 


0 
as.factor(subject):pos 174 29.6740 0.17054 1.3158 0.005541997 
Residuals 1489 192.9879 0.12961 


Regression model for significant main effects & interactions of log(response time) 


xxx Tinear Model *** 


Call: lm(formula = log(resp.time) ~ as.factor(subject) * status + 
as.factor(subject) * pos, data = sam2) 
Residuals: 

Min 10 Median 30 Max 


-1.865 -0.208 -0.02773 0.187 1.418 


Coefficients: 

Value Std. Error t value Pr(>|t|) 

(Intercept) 0.8497 0.1128 7.5357 0.0000 
as.factor(subject)2 0.0561 0.1537 0.3649 0.7152 
as.factor(subject)3 0.4518 0.1504 3.0035 0.0027 
as.factor(subject)4 0.5438 0.1514 3.5911 0.0003 
as.factor(subject)5 0.5683 0.1458 3.8984 0.0001 
as.factor(subject)6 0.1629 0.1516 1.0748 0.2826 
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(output edited for space) 


as.factor(subject)30 0.4665 





























0.1545 3.0188 0.0026 

status SOV4026 0.1208 -3.3333 [x 

posH 0.1402 0.2269 0.6180 0.5367 

posL 0.4812 0.1896 2.5385 0.0112 

posLA 1.1118 0.2701 4.1159 0.0000 

posN 0.5011 0.1896 2.6433 0.0083 

posP 0.8548 0.2784 3.0703 0.0022 

posR 0.5190 0.1424 3.6443 0.0003 

as.factor(subject)2status 0.4240 0.1552 2.7311 0.0064 

as.factor(subject)3status 0.1652 0.1578 1.0469 0.2953 

as.factor(subject)4status 0.3902 0.1572 2.4816 0.0132 
(output edited for space) 

as.factor(subject)29status 0.3882 0.1555 2.4964 0.0127 

as.factor (subject) 30status 0.3215 0.1575 2.0405 0.0415 

as.factor(subject)2posH 0.5219 0.3199 1.6314 0.1030 

as.factor(subject)3posH 0.2168 0.3201 0.6772 0.4984 

as.factor(subject)4posH 0.3699 0.3187 1.1606 0.2460 

as.factor(subject)5posH 0.1302 0.3187 0.4087 0.6828 

as.factor(subject)6posH 0.0571 0.3534 0.1617 0.8716 

as.factor (subject) 7posH -0.1477 0.3194 -0.4624 0.6439 

as.factor(subject) 8posH 0.1143 0.3187 0.3586 0.7199 

as.factor(subject)9posH 0.1372 0.3187 0.4304 0.6669 
(output edited for space) 

as.factor(subject)29posR -0.3530 0.1842 =1.9166 0.0555 

as.factor (subject) 30posR -0.3953 0.1863 -2.1215 0.0340 


Residual standard error: 0.36 on 1489 degrees of freedom 
F-statistic: 6.722 on 239 and 1489 degrees of freedom, the p-value is 0 


One-sided t-test for difference in response rates 


> t.test (sam3Sresp.rate[sam3Sstatus == 1], 
sam3Sresp.rate[sam3Sstatus == 0], alt = "greater") 


Standard Two-Sample t-Test 


data: sam3Sresp.rate[sam3S$status == 1] and 
sam3Sresp.rate[sam3Sstatus == 0] 

t = 3.7646, df = 1726, pevalue)=00007 

alternative hypothesis: true difference in means is greater than 0 

sample estimates: 

mean of x mean of y 
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ANOVA model for main effects of log(response rate) 


> anova (lm (log(resp.rate) ~ as.factor(subject) + status 
data = sam3) ) 
Analysis of Variance Table 


Response: log(resp.rate) 


Terms added sequentially (first to last) 
D£ Sum of Sq Mean Sq F Value 
as.factor (subject) 29 93.114 3.21084 2.45490 
status 1 14.333 14.33284 10.95838 
pos 6 292.115 48.68575 37.22340 
Residuals 1691 2211.716 1.30793 


Regression model for main effects of log(response rate) 


xxx Tinear Model *** 














Call: lm(formula = log(resp.rate) ~ as.factor(subject) + s 
data = sam3) 
Residuals: 
Min 1Q Median 30 Max 
-7.111 -0.6954 0.2052 0.839 2.766 
Coefficients: 
Value Std. Error t value Pr(>/tl) 
(Intercept) 1.9459 0.1787 10.8870 0.0000 
as.factor(subject)2 -0.5409 0.2279 =2; 3733 0.0177 
as.factor(subject)3 -0.0485 0.2324 -0.2088 0.8346 
as.factor(subject)4 -0.5110 0.2305 -2.2170 0.0268 
as.factor(subject)5 -0.5263 0.2279 =2,3093 0.0210 
as.factor(subject)6 -0.2717 0.2297 -1.1825 0..23°72 
as.factor(subject)7 -0.5139 0.2296 -2.2379 0.0254 
as.factor (subject) 8 0.0774 0.2271 0.3407 0.7334 
as.factor(subject)9 -0.3337 0.2279 -1.4640 0.1434 
(output edited for space) 
as.factor (subject) 28 0.0174 0.2288 0.0762 0.9393 
as.factor(subject)29 -0.2262 0.2280 -0..9921 0.3213 
as.factor(subject)30 -0.2618 0.2297 -1.1397 0.2546 
status 0.0552 3.3916 
posH -0.9089 0.1319 -—6.8920 0.0000 
posL -0.0900 0.1160 -0.7758 0.4380 
posLA 1.2245 0.1105 11.0830 0.0000 
posN 0.3938 0.0947 4.1595 0.0000 
posP 0.5005 On 1339 3.7369 0.0002 
posR 0.1294 0.0687 1.8816 0.0601 
Residual standard error: 1.144 on 1691 degrees of freedom 








+ pos, 


Pr (F) 
0.0000276876 


0.0000000000 


tatus + pos, 


F-statistic: 8.486 on 36 and 1691 degrees of freedom, the p-value is 0 
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Final ANOVA model for main effects & interactions of log(response rate) 








> anova (1m (log (resp.rate) a as.factor (subject) - status 
as.factor(subject) * pos, data = sam3)) 
Analysis of Variance TableRespons log(resp.rate) 
Terms added sequentially (first to last) 
Df Sum of Sq Mean Sq F Value Pr (F) 
as.factor (subject) 29 93.114 3.21084 2.54637 0.0000128 
status 1 14.333 14.33284 11.36672 
pos 6 292.115 48.68575 38.61044 0.0000000 
as.factor (subject) :status 29 41.850 1.44309 1.14445 0.2728819 
as.factor(subject):pos 174 293.576 1.68722 1.33805 0.0034754 
Residuals 1488 1876.291 1.26095 


Regression model for significant main effects & interactions of log(response rate) 


*x*x*x Tinear Model *** 


Call: Ilm(formula = log(resp.rate) 
data = sam3) 
Residuals: 

Min 1Q Median 3Q Max 


-7.089 -0.6257 0.1336 0.7465 3.434 


~ as.factor (subject) 


Value Std. Error t value Pr(>|t]) 














Coefficients: 

(Intercept) 1.6941 0.283 
as.factor(subject)2 -0.6903 0.391 
as.factor(subject)3 0.3907 0.391 
as.factor(subject)4 -0.1628 0.381 
as.factor(subject)5 -0.2654 0.381 
as.factor(subject)6 0.6621 0.381 
as.factor (subject) 7 -0.2796 0.386 
as.factor(subject)8 0.2768 0.381 
as.factor(subject)9 0.1923 0.381 

as.factor(subject)10 -0.1443 0.381 

as.factor(subject)11 -0.3660 0.381 

as.factor(subject)12 -0.6473 0.381 

as.factor(subject)13 0.2806 0.381 
(output edited for space) 

posH -0.9708 0.707 

posL 0.4223 0.576 

posLA 1.3045 0.843 

posN 0.7188 0.576 

posP 1.1647 0.844 

posR 0.6517 0.440 

status 0.056 

as.factor(subject)2posH 1.3695 0.998 

as.factor (subject) 3posH -0.4429 0.998 

as.factor (subject) 4posH -0.6176 0.994 

as.factor(subject)5posH 0.6545 0.994 
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0) 
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WANA DAND UW D~>D OO 


5.9865 
. 7620 
0.9972 
~4265 
.6954 
1.7349 
~7234 
0.7253 
0.5037 
3782 
9588 
- 6962 
0.7350 


=1,.39719 
0.7323 
1.5464 
1.2466 
1.3797 
1.4790 
3.1660 
so LT 
- 4436 
.6210 
0.6581 





0.0000 
0.0783 
0.3188 
0.6698 
0.4869 
0.0830 
0.4696 
0.4684 
0.6145 
0.7053 
0.3378 
0.0900 
0.4624 





* pos + status, 


+ 


(output edited for space) 


as.factor(subject)29posR -0.5769 0.5721 -1.0083 0.3135 
as.factor (subject) 30posR -0.6989 0.5789 =—1.2073 0.2275 


Residual standard error: 1.125 on 1517 degrees of freedom 
F-statistic: 2.606 on 210 and 1517 degrees of freedom, the p-value is 0 


T-test for difference in treatments 


> t.test (sam2Sdiff[sam2Streatment == 2], 
sam2Sdiff[sam2Streatment == 1]) 

Standard Two-Sample t-—Test 
data: sam2Sdiff[sam2Streatment == 2] and 


sam2$diff[sam2$treatment == 1] 
t = 2.3494, df = 28, pavalluel=(0U0Z6L 
alternative hypothesis: true difference in means is not equal to 0 
sample estimates: 
mean of x mean of y 
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APPENDIX J. SUBJECTIVE RATINGS OF SITUATIONAL 
AWARENESS 


This chart shows the subjective rating of situational awareness by participants 
during spatialized and non-spatialized segments of study. Also included is a subjective 


rating of radio call clarity. 


_pubiert | spatialized | non-spatialized 





8.63 6.17 7.10 
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APPENDIX K. PARTICIPANT COMMENTS 


Very realistic simulation. 

Spatial has a plus in resolving locations of a/c. 

| think the spatialized sound provided good azimuthal cues, but not necessarily elevation cues. | believe 
spatialized auditory cueing was very helpful during my experiment. 

My response times for non-spatialized portion seemed to be less. This could be partially due to the fact 
that it is an unfamiliar field to me. Once the spatialized sound was turned on, sa was enhanced 10 fold. 
Knowledge of the surrounding areas was not necessary to locate calling aircraft. 

The spatialized sound definitely helped in initially acquiring the aircraft if it was an initial call or | was not 
sure where they were in the pattern. The more | got used to using it and having it help me direct my 
eyes the more my SA improved. 

Spatialized portion seemed significantly easier to locate subject when part of radio transmission was not 
completely heard. Head-tracked spatialized audio was most effective when the controller is constantly 
moving his / her head if stationary when a call comes in; must move the head to discriminate fore / aft 
differences. After a while started continuously moving head -- radio calls received then were very quickly 
discriminated and located. 

| responded better on the non-spatialized segment because | had become much better oriented in the 
spatialized segment. If | had done the non-spatialized segment first, | think | would have understood 
much less and been more disoriented. 

Very cool graphics. Spatialized portion -- was very helpful when Knightrider called left abeam & Dragon 
called for the midfield break to hear it in the right ear (plus beeping for north initial). Really helpful to 
orient you. 

During spatialized portion | relied more heavily on the sound to guide me to the target and actually paid 
less attention to the content of the transmission, particularly call signs. During non-spatialized phase | 
relied more heavily on the content of the call to locate the target and felt | spent more time scanning the 
airspace and trying to sort call signs so | thought my overall SA was a bit higher. 

Much easier to manage traffic with spatialized sound. | could look where the transmission was coming 
from and anticipate the aircraft's request based on its position. 

Spatialized helped with cueing during the transmission, especially when an unexpected call (near 
aircraft) was received. 


Spatialized - hope all towers get this someday. 
It was clear to me that spatializing the sound enhanced my situational awareness. 


Spatialized sound made a big difference in ability to quickly and accurately localize the source of a 
transmission. It helped make up for unfamiliarity with the operating area and unfamiliarity with typical 
radio calls used at the operating area. 

Interesting. Would be nice to have spatialized sound in the cockpit. Definitely improves your SA when 
you have a busy lookout duty. 

| felt very little SA difference from spatialized to non-spatialized. This seems to be more from training 
(lack of) than cues available. The sound cue was there at beginning of transmission but | found myself 
not determining position until partially through transmission. 

Spatialized sound helped. Quicker to spot aircraft. Sound gave general direction to look before radio 
call told you their position. 

| tended not to memorize call signs of the aircraft but rely on position calls and the spatialized direction of 
arrival to find the aircraft as they called for clearances. 

Distance dependent radio calls could be confused with a bad transmitter otherwise the idea is great. 
Great idea, | felt much more aware of the aircraft. 

| could hear and understand the radio calls better during non-spatialized portion. | could definitely turn 
my head quicker and locate the trarget faster during the spatialized portion (however, | had more trouble 
understanding). 





65 


THIS PAGE INTENTIONALLY LEFT BLANK 


66 


LIST OF REFERENCES 


Azuma, R., Daily, M., & Krozel, J. (1996). Advanced Human-Computer Interfaces for 
Air Traffic Management and Simulation. American Institute of Aeronautics and 
Astronautics. 


Begault, D. (1993). Call Sign Intelligibility Improvement Using a Spatial Auditory 
Display. Hanover, MD: NASA Center for Aerospace Information. (NASA/TM-104014) 


Begault, D. (1999). Auditory and Non-auditory Factors That Potentially Influence Virtual 
Acoustic Imagery. Proceedings of the Audio Engineering Society 16th International 
Conference on Spatial Sound Reproduction. 


Begault, D. (2000). 3-D Sound for Virtual Reality and Multimedia. Hanover, MD: 
NASA Center for Aerospace Information. (NASA/TM — 2000 — 209606) 


Begault, D., Wenzel, E., & Anderson, M. (2001). Direct Comparison of the Impact of 
Head Tracking, Reverberation, and Individualized Head-related Transfer Functions on 
the Spatial Perception of a Virtual Speech Source. Journal of the Audio Engineering 
Society. 


Campbell, J. (2002). The Effect of Sound Spatialization on Responses to Overlapping 
Messages. Master’s Thesis. Naval Postgraduate School. 


CNN. (2002). Many Feared Dead in Midair Crash. 
www.cnn.com/2002/WORLD/europe/07/01/plane.collision 





Dickie, A. (2002). Modeling Robot Swarms Using Agent-Based Simulation. Master’s 
Thesis. Naval Postgraduate School. 


Loomis, J., Hebert, C., & Cincinelli, J. (1990). Active Localization of Virtual Sounds. 
Journal of the Acoustical Society of America. 


McKinley, M. & Ericson, M. (1995). In Flight Demonstration of a 3-D Auditory Display. 
Binaural and Spatial Hearing in Real and Virtual Environments. Gilkey, R. & 
Anderson, T. (Eds.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. 


Mackensen, P., Fruhmann, M., Thanner, M., Theile, G., Horbach, U., & 
Karamustafaoglu, A. (2000). Head-Tracker Based Auralization Systems: Additional 
Considerations of Vertical Head Movements. Journal of the Audio Engineering Society. 


Metzger, U. (2001). The Role of the Air Traffic Controller in Future Air Traffic 


Management: An Empirical Study of Active Control Versus Passive Monitoring. Human 
Factors. 


67 


Moller, H., Sorensen, M., Hammershoi, D., & Jensen, C. (1995). Head-related Transfer 
Functions of Human Subjects. Journal of the Audio Engineering Society. 


Perrott, D., Cisneros, J., McKinley, R., & D’Angelo, W. (1996). Aurally Aided Visual 
Search under Virtual and Free-Field Listening Conditions. Human Factors. 


Shilling, R., Letowski, T., & Storms R. (2000). Spatial Auditory Displays for Use Within 
Attack Rotary Wing Aircraft. Proceedings of the International Conference on Auditory 
Displays. 


Shilling, R. & Shinn-Cunningham, B. (2002). Virtual Auditory Displays. In Handbook 
for Virtual Environment Technology. Stanney, K. (Ed.). New York: Lawrence Erlbaum, 
Associates, Inc. 


Speith, W., Curtis, J. & Webster, J. (1954). Responding to One of Two Simultaneous 
Messages. The Journal of the Acoustical Society of America. 


Thompson, P. & Webster, J. (1963). The Effect of Talker-Listener Angle on Word 
Intelligibility. Acustica. 


Wenzel, E. (1992). Localization in Virtual Acoustic Displays. Presence: Teleoperators 
and Virtual Environments. 


Webster, J. & Thompson, P. (1953). Responding to Both of Two Overlapping Messages. 
The Journal of the Acoustical Society of America. 


Wenzel, E., Arruda, M., Kistler, D. & Wightman F. (1993). Localization Using 
Nonindividualized Head-related Transfer Functions. Journal of the Acoustical Society of 
America. 


Wenzel, E., & Begault, D. (1995). Terminal Area Productivity (TAP) Program—Taxi 
Navigation and Situation Awareness (T-NASA) System: 3-D Audio Ground Collision 
Avoidance System (GCAS) and Navigation System. http://human-factors.arc.nasa.gov/, 
24 August 2002. 





Wenzel, E., & Begault, D. (1998). Virtual Acoustic Enhancement of Traffic Collision 
Avoidance System Advisories. http://human-factors.arc.nasa.gov/, 24 August 2002. 





Wightman, F., Kistler, D., & Perkins, M. (1987). A New Approach to the Study of 
Human Sound Localization. In Directional Hearing. Yost, W. & Gourevitch, G. (Eds.). 
New York: Springer-Verlag. 


68 


10. 


A. 


12. 


INITIAL DISTRIBUTION LIST 


Defense Technical Information Center 
Ft. Belvoir, Virginia 


Dudley Knox Library 
Naval Postgraduate School 
Monterey, California 


Marine Corps Representative 
Naval Postgraduate School 
Monterey, California 


Director, Training and Education, MCCDC, Code 46 
Quantico, Virginia 


Director, Marine Corps Research Center, MCCDC, Code C40RC 
Quantico, Virginia 


Marine Corps Tactical Systems Support Activity (Attn: Operations Officer) 
Camp Pendleton, California 


Director, Studies and Analysis Division, MCCDC, Code 45 
Quantico, Virginia 


Marine Corps Air Station Camp Pendleton (Attn: Operations Officer) 
Camp Pendleton, California 


LCDR Russell D. Shilling, USN 
Code 32 

Naval Postgraduate School 
Monterey, California 


LtCol Gregory K. Mislick, USMC 
Code 30 

Naval Postgraduate School 
Monterey, California 


Ms. Agnieszka Roginska 
5274 Scotts Valley Drive, Unit G 
Scotts Valley, California 


GS12 Victoria Evans 
Marine Corps Air Station Camp Pendleton 
Camp Pendleton, California 


69 


